Introduction to Computer Vision: Plant Seedlings Classification

Problem Statement

Context

In recent times, the field of agriculture has been in urgent need of modernizing, since the amount of manual work people need to put in to check if plants are growing correctly is still highly extensive. Despite several advances in agricultural technology, people working in the agricultural industry still need to have the ability to sort and recognize different plants and weeds, which takes a lot of time and effort in the long term. The potential is ripe for this trillion-dollar industry to be greatly impacted by technological innovations that cut down on the requirement for manual labor, and this is where Artificial Intelligence can actually benefit the workers in this field, as the time and energy required to identify plant seedlings will be greatly shortened by the use of AI and Deep Learning. The ability to do so far more efficiently and even more effectively than experienced manual labor, could lead to better crop yields, the freeing up of human inolvement for higher-order agricultural decision making, and in the long term will result in more sustainable environmental practices in agriculture as well.

Objective

The aim of this project is to Build a Convolutional Neural Netowrk to classify plant seedlings into their respective categories.

Data Dictionary

The Aarhus University Signal Processing group, in collaboration with the University of Southern Denmark, has recently released a dataset containing images of unique plants belonging to 12 different species.

  • The dataset can be download from Olympus.
  • The data file names are:
    • images.npy
    • Labels.csv
  • Due to the large volume of data, the images were converted to the images.npy file and the labels are also put into Labels.csv, so that you can work on the data/project seamlessly without having to worry about the high data volume.

  • The goal of the project is to create a classifier capable of determining a plant's species from an image.

List of Species

  • Black-grass
  • Charlock
  • Cleavers
  • Common Chickweed
  • Common Wheat
  • Fat Hen
  • Loose Silky-bent
  • Maize
  • Scentless Mayweed
  • Shepherds Purse
  • Small-flowered Cranesbill
  • Sugar beet

Note: Please use GPU runtime on Google Colab to execute the code faster.

Importing necessary libraries

In [2]:
# Installing the libraries with the specified version.
# uncomment and run the following line if Google Colab is being used
# !pip install tensorflow==2.15.0 scikit-learn==1.2.2 seaborn==0.13.1 matplotlib==3.7.1 numpy==1.25.2 pandas==1.5.3 opencv-python==4.8.0.76 -q --user
In [ ]:
# Installing the libraries with the specified version.
# uncomment and run the following lines if Jupyter Notebook is being used
#!pip install tensorflow==2.13.0 scikit-learn==1.2.2 seaborn==0.11.1 matplotlib==3.3.4 numpy==1.24.3 pandas==1.5.2 opencv-python==4.8.0.76 -q --user

Note: After running the above cell, kindly restart the notebook kernel and run all cells sequentially from the start again.

In [3]:
import os
import numpy as np                                                                               # Importing numpy for Matrix Operations
import pandas as pd                                                                              # Importing pandas to read CSV files
import matplotlib.pyplot as plt                                                                  # Importting matplotlib for Plotting and visualizing images
import math                                                                                      # Importing math module to perform mathematical operations
import cv2                                                                                       # Importing openCV for image processing
import seaborn as sns                                                                            # Importing seaborn to plot graphs
import random

# Tensorflow modules
import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator                              # Importing the ImageDataGenerator for data augmentation
from tensorflow.keras.models import Sequential                                                   # Importing the sequential module to define a sequential model
from tensorflow.keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D,BatchNormalization # Defining all the layers to build our CNN Model
from tensorflow.keras.optimizers import Adam,SGD                                                 # Importing the optimizers which can be used in our model
from sklearn import preprocessing                                                                # Importing the preprocessing module to preprocess the data
from sklearn.model_selection import train_test_split                                             # Importing train_test_split function to split the data into train and test
from sklearn.metrics import confusion_matrix                                                     # Importing confusion_matrix to plot the confusion matrix

# Display images using OpenCV
from google.colab.patches import cv2_imshow                                                      # Importing cv2_imshow from google.patches to display images

# Ignore warnings
import warnings
warnings.filterwarnings('ignore')

Loading the dataset

In [5]:
# Uncomment and run the below code if you are using google colab
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
In [6]:
# Load the image file of the dataset
images = np.load('/content/drive/MyDrive/Python/images.npy')

# Load the labels file of the dataset
labels = pd.read_csv('/content/drive/MyDrive/Python/Labels.csv')

Data Overview

Understand the shape of the dataset

In [7]:
print(images.shape)
(4750, 128, 128, 3)

There are 4750 images in the dataset, each of shape 128x128 and 3 Color channels(RGB)

In [8]:
print(labels.shape)
(4750, 1)

there are 4750 labels in the dataset, each label representing a single value.

Plotting images using OpenCV and matplotlib

In [9]:
#Plotting an image from the dataset using OpenCV
cv2_imshow(images[10])
In [10]:
#Plotting an image from the dataset using matplotlib
plt.imshow(images[10])
Out[10]:
<matplotlib.image.AxesImage at 0x7a4d8d51b990>

We can observe that the images are being shown in different colors when plotted with openCV and matplotlib as OpenCV reads images in BGR format and this shows that the given numpy arrays were generated from the original images using OpenCV. These will need to be converted from BGR images to RGB images so we could interpret them easily.

Exploratory Data Analysis

  • EDA is an important part of any project involving data.
  • It is important to investigate and understand the data better before building a model with it.
  • A few questions have been mentioned below which will help you understand the data better.
  • A thorough analysis of the data, in addition to the questions mentioned below, should be done.
  1. How are these different category plant images different from each other?
  2. Is the dataset provided an imbalance? (Check with using bar plots)

Convert the BGR images to RGB images.

In [11]:
# Converting the images from BGR to RGB using cvtColor function of OpenCV
#We will do this conversion before data pre-processeing  to help with Visual EDA
for i in range(len(images)):
  images[i] = cv2.cvtColor(images[i], cv2.COLOR_BGR2RGB)

Display Samples of every class

In [17]:
#Plotting an image from each of the classes while printing  corresponding label

unique_classes = labels['Label'].unique()

# Create a figure to display 12 images
fig, axes = plt.subplots(3, 4, figsize=(10, 8))
fig.suptitle('Image from Each Class', fontsize=16)

for i, class_name in enumerate(unique_classes):
    # Get all images of the current class
    class_images = images[labels['Label'] == class_name]

    # Randomly select one image from the class
    random_image = class_images[random.randint(0, class_images.shape[0] - 1)]

    # Plot the selected image
    ax = axes[i // 4, i % 4]
    ax.imshow(random_image)
    ax.set_title(class_name)
    ax.axis('off')

plt.tight_layout()
plt.show()

Class Distribution

In [8]:
#Countplot of classes to see the data spread/imbalance
sns.countplot(labels['Label'])
plt.xticks(rotation='vertical')
Out[8]:
(array([  0., 100., 200., 300., 400., 500., 600., 700.]),
 [Text(0.0, 0, '0'),
  Text(100.0, 0, '100'),
  Text(200.0, 0, '200'),
  Text(300.0, 0, '300'),
  Text(400.0, 0, '400'),
  Text(500.0, 0, '500'),
  Text(600.0, 0, '600'),
  Text(700.0, 0, '700')])
  • As we can see from the above plot, the dataset is imbalanced

Visual Inspection

In [24]:
categories = labels['Label'].unique()
fig, axes = plt.subplots(len(categories), 3, figsize=(20, 20))

for i, category in enumerate(categories):
    category_images = images[labels['Label'] == category]
    for j in range(3):
        random_index = np.random.randint(0, len(category_images))
        axes[i, j].imshow(category_images[random_index])
        axes[i, j].axis('off')
        if j == 0:
           axes[i, j].set_ylabel(category, fontsize=14)
           axes[i, j].set_title(f'{category} image {j+1}', fontsize=10)
plt.subplots_adjust(wspace=0.1, hspace=0.1)
plt.tight_layout()
plt.show()

EDA Observations

  • Images show clear variations in terms of plant structure,leaf shape and structure,color ,texture and pattern. These distinct visual characteristics (color,structure,appearenace,shape ,texture and pattern) can be leveraged for classification.
  • Size and proportions of the plants within the images vary which will require normalisation to offset the variations for the model input
  • Class Imbalance is observed , some categories having larger dataset than others and this will need to be addressed for building an effective model

Data Pre-Processing

Resize the images

As the size of the images is large, it may be computationally expensive to train on these larger images; therefore, it is preferable to reduce the image size from 128 to 64.

In [12]:
#Plot an images before resizing the images
plt.imshow(images[3])
Out[12]:
<matplotlib.image.AxesImage at 0x7a4d7c50dcd0>
In [13]:
#Resize the images
images_decreased=[]
height = 64
width = 64
dimensions = (width, height)
for i in range(len(images)):
  images_decreased.append( cv2.resize(images[i], dimensions, interpolation=cv2.INTER_LINEAR))
In [14]:
#Plot the same image after resizing
plt.imshow(images_decreased[3])
Out[14]:
<matplotlib.image.AxesImage at 0x7a4d7c561590>

Data Preparation for Modeling

  • Before you proceed to build a model, you need to split the data into train, test, and validation to be able to evaluate the model that you build on the train data
  • You'll have to encode categorical features and scale the pixel values.
  • You will build a model using the train data and then check its performance

Split the dataset

As there are less images in the dataset, we will use 10% of our data for testing, 10% of data for validation and 80% of data for training.

In [15]:
from sklearn.model_selection import train_test_split
X_temp, X_test, y_temp, y_test = train_test_split(np.array(images_decreased),labels , test_size=0.1, random_state=42,stratify=labels)
X_train, X_val, y_train, y_val = train_test_split(X_temp,y_temp , test_size=0.111, random_state=42,stratify=y_temp)
In [16]:
print(X_train.shape,y_train.shape)
print(X_val.shape,y_val.shape)
print(X_test.shape,y_test.shape)
(3800, 64, 64, 3) (3800, 1)
(475, 64, 64, 3) (475, 1)
(475, 64, 64, 3) (475, 1)

Encode the target labels

In [17]:
# Convert labels from names to one hot vectors using Labelbinariser

from sklearn.preprocessing import LabelBinarizer
enc = LabelBinarizer()
y_train_encoded = enc.fit_transform(y_train)
y_val_encoded=enc.transform(y_val)
y_test_encoded=enc.transform(y_test)

Data Normalization

In [18]:
# Normalizing the image pixels
X_train_normalized = X_train.astype('float32')/255.0
X_val_normalized = X_val.astype('float32')/255.0
X_test_normalized = X_test.astype('float32')/255.0

Model Building

In [19]:
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()
In [20]:
# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [26]:
# Intializing a sequential model
model = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension of images
model.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))

# Adding max pooling to reduce the size of output of first conv layer
model.add(MaxPooling2D((2, 2), padding = 'same'))

model.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model.add(MaxPooling2D((2, 2), padding = 'same'))

# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model.add(Dense(16, activation='relu'))
model.add(Dropout(0.3))
# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model.add(Dense(12, activation='softmax'))

# Using Adam Optimizer
opt=Adam()
# Compile model
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy','recall','f1_score','precision'])

# Generating the summary of the model
model.summary()
Model: "sequential_2"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                          Output Shape                         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ conv2d_4 (Conv2D)                    │ (None, 64, 64, 64)          │           1,792 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_4 (MaxPooling2D)       │ (None, 32, 32, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_5 (Conv2D)                    │ (None, 32, 32, 32)          │          18,464 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_5 (MaxPooling2D)       │ (None, 16, 16, 32)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten_2 (Flatten)                  │ (None, 8192)                │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_4 (Dense)                      │ (None, 16)                  │         131,088 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dropout_2 (Dropout)                  │ (None, 16)                  │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_5 (Dense)                      │ (None, 12)                  │             204 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
 Total params: 151,548 (591.98 KB)
 Trainable params: 151,548 (591.98 KB)
 Non-trainable params: 0 (0.00 B)
In [27]:
history_1 = model.fit(
            X_train_normalized, y_train_encoded,
            epochs=50,
            validation_data=(X_val_normalized,y_val_encoded),
            batch_size=32,
            verbose=2
)
Epoch 1/50
119/119 - 10s - 81ms/step - accuracy: 0.1203 - f1_score: 0.0600 - loss: 2.4411 - precision: 0.0000e+00 - recall: 0.0000e+00 - val_accuracy: 0.1811 - val_f1_score: 0.0755 - val_loss: 2.3750 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00
Epoch 2/50
119/119 - 3s - 28ms/step - accuracy: 0.2174 - f1_score: 0.1229 - loss: 2.3178 - precision: 0.5217 - recall: 0.0032 - val_accuracy: 0.3347 - val_f1_score: 0.1361 - val_loss: 2.1474 - val_precision: 1.0000 - val_recall: 0.0021
Epoch 3/50
119/119 - 1s - 6ms/step - accuracy: 0.2966 - f1_score: 0.1901 - loss: 2.0917 - precision: 0.6941 - recall: 0.0466 - val_accuracy: 0.3853 - val_f1_score: 0.1942 - val_loss: 1.8599 - val_precision: 0.7692 - val_recall: 0.1053
Epoch 4/50
119/119 - 1s - 6ms/step - accuracy: 0.3392 - f1_score: 0.2198 - loss: 1.8968 - precision: 0.6348 - recall: 0.0855 - val_accuracy: 0.4316 - val_f1_score: 0.2526 - val_loss: 1.6840 - val_precision: 0.7059 - val_recall: 0.1263
Epoch 5/50
119/119 - 1s - 6ms/step - accuracy: 0.3697 - f1_score: 0.2600 - loss: 1.8048 - precision: 0.6722 - recall: 0.1182 - val_accuracy: 0.4884 - val_f1_score: 0.3084 - val_loss: 1.5809 - val_precision: 0.6899 - val_recall: 0.1874
Epoch 6/50
119/119 - 1s - 7ms/step - accuracy: 0.3913 - f1_score: 0.3017 - loss: 1.7125 - precision: 0.7030 - recall: 0.1526 - val_accuracy: 0.5011 - val_f1_score: 0.3431 - val_loss: 1.5172 - val_precision: 0.7398 - val_recall: 0.1916
Epoch 7/50
119/119 - 1s - 11ms/step - accuracy: 0.4176 - f1_score: 0.3324 - loss: 1.6157 - precision: 0.6928 - recall: 0.1882 - val_accuracy: 0.5116 - val_f1_score: 0.3593 - val_loss: 1.4461 - val_precision: 0.7246 - val_recall: 0.2105
Epoch 8/50
119/119 - 1s - 11ms/step - accuracy: 0.4405 - f1_score: 0.3514 - loss: 1.5545 - precision: 0.7275 - recall: 0.2171 - val_accuracy: 0.5474 - val_f1_score: 0.4052 - val_loss: 1.4251 - val_precision: 0.7821 - val_recall: 0.2568
Epoch 9/50
119/119 - 1s - 7ms/step - accuracy: 0.4613 - f1_score: 0.3706 - loss: 1.4948 - precision: 0.7087 - recall: 0.2382 - val_accuracy: 0.5747 - val_f1_score: 0.4251 - val_loss: 1.3572 - val_precision: 0.8158 - val_recall: 0.2611
Epoch 10/50
119/119 - 1s - 6ms/step - accuracy: 0.4700 - f1_score: 0.3691 - loss: 1.4834 - precision: 0.7253 - recall: 0.2397 - val_accuracy: 0.5705 - val_f1_score: 0.4302 - val_loss: 1.3138 - val_precision: 0.8075 - val_recall: 0.2737
Epoch 11/50
119/119 - 1s - 6ms/step - accuracy: 0.4821 - f1_score: 0.3794 - loss: 1.4288 - precision: 0.7298 - recall: 0.2737 - val_accuracy: 0.5705 - val_f1_score: 0.4367 - val_loss: 1.2931 - val_precision: 0.7692 - val_recall: 0.2947
Epoch 12/50
119/119 - 1s - 11ms/step - accuracy: 0.4895 - f1_score: 0.3871 - loss: 1.4138 - precision: 0.7184 - recall: 0.2800 - val_accuracy: 0.6189 - val_f1_score: 0.4838 - val_loss: 1.2311 - val_precision: 0.7391 - val_recall: 0.3221
Epoch 13/50
119/119 - 1s - 11ms/step - accuracy: 0.4955 - f1_score: 0.4016 - loss: 1.4028 - precision: 0.7155 - recall: 0.2774 - val_accuracy: 0.5853 - val_f1_score: 0.4540 - val_loss: 1.2794 - val_precision: 0.7957 - val_recall: 0.3116
Epoch 14/50
119/119 - 1s - 6ms/step - accuracy: 0.5045 - f1_score: 0.4062 - loss: 1.3387 - precision: 0.7382 - recall: 0.3087 - val_accuracy: 0.6042 - val_f1_score: 0.4573 - val_loss: 1.2118 - val_precision: 0.8352 - val_recall: 0.3095
Epoch 15/50
119/119 - 1s - 11ms/step - accuracy: 0.5137 - f1_score: 0.4271 - loss: 1.3193 - precision: 0.7382 - recall: 0.3205 - val_accuracy: 0.6168 - val_f1_score: 0.4912 - val_loss: 1.1546 - val_precision: 0.7767 - val_recall: 0.3516
Epoch 16/50
119/119 - 1s - 10ms/step - accuracy: 0.5155 - f1_score: 0.4194 - loss: 1.3014 - precision: 0.7435 - recall: 0.3250 - val_accuracy: 0.5768 - val_f1_score: 0.4614 - val_loss: 1.2506 - val_precision: 0.7536 - val_recall: 0.3284
Epoch 17/50
119/119 - 1s - 11ms/step - accuracy: 0.5316 - f1_score: 0.4412 - loss: 1.2777 - precision: 0.7473 - recall: 0.3400 - val_accuracy: 0.5705 - val_f1_score: 0.4633 - val_loss: 1.2727 - val_precision: 0.7853 - val_recall: 0.3158
Epoch 18/50
119/119 - 1s - 11ms/step - accuracy: 0.5316 - f1_score: 0.4427 - loss: 1.2716 - precision: 0.7592 - recall: 0.3511 - val_accuracy: 0.6084 - val_f1_score: 0.4818 - val_loss: 1.1845 - val_precision: 0.7833 - val_recall: 0.3347
Epoch 19/50
119/119 - 1s - 8ms/step - accuracy: 0.5421 - f1_score: 0.4539 - loss: 1.2595 - precision: 0.7511 - recall: 0.3439 - val_accuracy: 0.6295 - val_f1_score: 0.5205 - val_loss: 1.1336 - val_precision: 0.7480 - val_recall: 0.4000
Epoch 20/50
119/119 - 1s - 10ms/step - accuracy: 0.5532 - f1_score: 0.4666 - loss: 1.2129 - precision: 0.7578 - recall: 0.3755 - val_accuracy: 0.6421 - val_f1_score: 0.5229 - val_loss: 1.1124 - val_precision: 0.7491 - val_recall: 0.4211
Epoch 21/50
119/119 - 1s - 9ms/step - accuracy: 0.5605 - f1_score: 0.4772 - loss: 1.1805 - precision: 0.7614 - recall: 0.3921 - val_accuracy: 0.6211 - val_f1_score: 0.5076 - val_loss: 1.1251 - val_precision: 0.7391 - val_recall: 0.4295
Epoch 22/50
119/119 - 1s - 11ms/step - accuracy: 0.5624 - f1_score: 0.4790 - loss: 1.1614 - precision: 0.7574 - recall: 0.4042 - val_accuracy: 0.6232 - val_f1_score: 0.5082 - val_loss: 1.0801 - val_precision: 0.7703 - val_recall: 0.4589
Epoch 23/50
119/119 - 1s - 10ms/step - accuracy: 0.5739 - f1_score: 0.4973 - loss: 1.1667 - precision: 0.7638 - recall: 0.4068 - val_accuracy: 0.6147 - val_f1_score: 0.5257 - val_loss: 1.1679 - val_precision: 0.7600 - val_recall: 0.4000
Epoch 24/50
119/119 - 1s - 10ms/step - accuracy: 0.5676 - f1_score: 0.4873 - loss: 1.1612 - precision: 0.7631 - recall: 0.3992 - val_accuracy: 0.6337 - val_f1_score: 0.5285 - val_loss: 1.1201 - val_precision: 0.7255 - val_recall: 0.4674
Epoch 25/50
119/119 - 1s - 7ms/step - accuracy: 0.5745 - f1_score: 0.4955 - loss: 1.1602 - precision: 0.7707 - recall: 0.4095 - val_accuracy: 0.6232 - val_f1_score: 0.5034 - val_loss: 1.0842 - val_precision: 0.7500 - val_recall: 0.4737
Epoch 26/50
119/119 - 1s - 10ms/step - accuracy: 0.5968 - f1_score: 0.5192 - loss: 1.1004 - precision: 0.7783 - recall: 0.4379 - val_accuracy: 0.6379 - val_f1_score: 0.5417 - val_loss: 1.0517 - val_precision: 0.7546 - val_recall: 0.5179
Epoch 27/50
119/119 - 1s - 6ms/step - accuracy: 0.5963 - f1_score: 0.5205 - loss: 1.0858 - precision: 0.7796 - recall: 0.4476 - val_accuracy: 0.6379 - val_f1_score: 0.5313 - val_loss: 1.0461 - val_precision: 0.7545 - val_recall: 0.5242
Epoch 28/50
119/119 - 1s - 7ms/step - accuracy: 0.5955 - f1_score: 0.5175 - loss: 1.1032 - precision: 0.7903 - recall: 0.4453 - val_accuracy: 0.6484 - val_f1_score: 0.5531 - val_loss: 1.0798 - val_precision: 0.7623 - val_recall: 0.5200
Epoch 29/50
119/119 - 1s - 6ms/step - accuracy: 0.6005 - f1_score: 0.5270 - loss: 1.0834 - precision: 0.7611 - recall: 0.4568 - val_accuracy: 0.6295 - val_f1_score: 0.5177 - val_loss: 1.0614 - val_precision: 0.7508 - val_recall: 0.5137
Epoch 30/50
119/119 - 1s - 12ms/step - accuracy: 0.5992 - f1_score: 0.5280 - loss: 1.0833 - precision: 0.7833 - recall: 0.4508 - val_accuracy: 0.6379 - val_f1_score: 0.5276 - val_loss: 1.0573 - val_precision: 0.7522 - val_recall: 0.5305
Epoch 31/50
119/119 - 1s - 11ms/step - accuracy: 0.5982 - f1_score: 0.5234 - loss: 1.1028 - precision: 0.7865 - recall: 0.4439 - val_accuracy: 0.6253 - val_f1_score: 0.5232 - val_loss: 1.0918 - val_precision: 0.7406 - val_recall: 0.4989
Epoch 32/50
119/119 - 1s - 7ms/step - accuracy: 0.6171 - f1_score: 0.5448 - loss: 1.0428 - precision: 0.7850 - recall: 0.4661 - val_accuracy: 0.6442 - val_f1_score: 0.5438 - val_loss: 1.0573 - val_precision: 0.7478 - val_recall: 0.5305
Epoch 33/50
119/119 - 1s - 6ms/step - accuracy: 0.6150 - f1_score: 0.5474 - loss: 1.0477 - precision: 0.7855 - recall: 0.4626 - val_accuracy: 0.6547 - val_f1_score: 0.5683 - val_loss: 1.0312 - val_precision: 0.7876 - val_recall: 0.5621
Epoch 34/50
119/119 - 1s - 11ms/step - accuracy: 0.6263 - f1_score: 0.5509 - loss: 1.0097 - precision: 0.7946 - recall: 0.4876 - val_accuracy: 0.6568 - val_f1_score: 0.5618 - val_loss: 1.0459 - val_precision: 0.7399 - val_recall: 0.5389
Epoch 35/50
119/119 - 1s - 7ms/step - accuracy: 0.6097 - f1_score: 0.5394 - loss: 1.0396 - precision: 0.7777 - recall: 0.4595 - val_accuracy: 0.6421 - val_f1_score: 0.5410 - val_loss: 1.0903 - val_precision: 0.7270 - val_recall: 0.5158
Epoch 36/50
119/119 - 1s - 7ms/step - accuracy: 0.6155 - f1_score: 0.5481 - loss: 1.0358 - precision: 0.7971 - recall: 0.4745 - val_accuracy: 0.6674 - val_f1_score: 0.5820 - val_loss: 1.0294 - val_precision: 0.7657 - val_recall: 0.5642
Epoch 37/50
119/119 - 1s - 10ms/step - accuracy: 0.6216 - f1_score: 0.5591 - loss: 1.0196 - precision: 0.7918 - recall: 0.4713 - val_accuracy: 0.6358 - val_f1_score: 0.5320 - val_loss: 1.0706 - val_precision: 0.7305 - val_recall: 0.5705
Epoch 38/50
119/119 - 1s - 10ms/step - accuracy: 0.6168 - f1_score: 0.5486 - loss: 1.0252 - precision: 0.7902 - recall: 0.4816 - val_accuracy: 0.6421 - val_f1_score: 0.5463 - val_loss: 1.0579 - val_precision: 0.7529 - val_recall: 0.5516
Epoch 39/50
119/119 - 1s - 6ms/step - accuracy: 0.6371 - f1_score: 0.5755 - loss: 0.9772 - precision: 0.7980 - recall: 0.5021 - val_accuracy: 0.6442 - val_f1_score: 0.5479 - val_loss: 1.1119 - val_precision: 0.7604 - val_recall: 0.5411
Epoch 40/50
119/119 - 1s - 11ms/step - accuracy: 0.6329 - f1_score: 0.5735 - loss: 0.9619 - precision: 0.7983 - recall: 0.5084 - val_accuracy: 0.6505 - val_f1_score: 0.5598 - val_loss: 1.1164 - val_precision: 0.7514 - val_recall: 0.5537
Epoch 41/50
119/119 - 1s - 6ms/step - accuracy: 0.6387 - f1_score: 0.5755 - loss: 0.9628 - precision: 0.8004 - recall: 0.5003 - val_accuracy: 0.6358 - val_f1_score: 0.5343 - val_loss: 1.1782 - val_precision: 0.7300 - val_recall: 0.5579
Epoch 42/50
119/119 - 1s - 7ms/step - accuracy: 0.6397 - f1_score: 0.5794 - loss: 0.9587 - precision: 0.8055 - recall: 0.5134 - val_accuracy: 0.6505 - val_f1_score: 0.5515 - val_loss: 1.1021 - val_precision: 0.7595 - val_recall: 0.5916
Epoch 43/50
119/119 - 1s - 8ms/step - accuracy: 0.6371 - f1_score: 0.5767 - loss: 0.9726 - precision: 0.7907 - recall: 0.5032 - val_accuracy: 0.6253 - val_f1_score: 0.5239 - val_loss: 1.1540 - val_precision: 0.7206 - val_recall: 0.5158
Epoch 44/50
119/119 - 1s - 8ms/step - accuracy: 0.6503 - f1_score: 0.5902 - loss: 0.9360 - precision: 0.8181 - recall: 0.5218 - val_accuracy: 0.6316 - val_f1_score: 0.5327 - val_loss: 1.1242 - val_precision: 0.7364 - val_recall: 0.5705
Epoch 45/50
119/119 - 1s - 7ms/step - accuracy: 0.6382 - f1_score: 0.5804 - loss: 0.9688 - precision: 0.8029 - recall: 0.5113 - val_accuracy: 0.6316 - val_f1_score: 0.5305 - val_loss: 1.1275 - val_precision: 0.7403 - val_recall: 0.5642
Epoch 46/50
119/119 - 1s - 7ms/step - accuracy: 0.6324 - f1_score: 0.5688 - loss: 0.9591 - precision: 0.7960 - recall: 0.5103 - val_accuracy: 0.6463 - val_f1_score: 0.5612 - val_loss: 1.0884 - val_precision: 0.7479 - val_recall: 0.5747
Epoch 47/50
119/119 - 1s - 10ms/step - accuracy: 0.6447 - f1_score: 0.5833 - loss: 0.9354 - precision: 0.8073 - recall: 0.5205 - val_accuracy: 0.6400 - val_f1_score: 0.5351 - val_loss: 1.0941 - val_precision: 0.7562 - val_recall: 0.5811
Epoch 48/50
119/119 - 1s - 11ms/step - accuracy: 0.6542 - f1_score: 0.5919 - loss: 0.9205 - precision: 0.8102 - recall: 0.5247 - val_accuracy: 0.6379 - val_f1_score: 0.5450 - val_loss: 1.1520 - val_precision: 0.7154 - val_recall: 0.5663
Epoch 49/50
119/119 - 2s - 14ms/step - accuracy: 0.6600 - f1_score: 0.6040 - loss: 0.9026 - precision: 0.8060 - recall: 0.5303 - val_accuracy: 0.6547 - val_f1_score: 0.5653 - val_loss: 1.1539 - val_precision: 0.7351 - val_recall: 0.5726
Epoch 50/50
119/119 - 2s - 18ms/step - accuracy: 0.6508 - f1_score: 0.5893 - loss: 0.9019 - precision: 0.8025 - recall: 0.5303 - val_accuracy: 0.6547 - val_f1_score: 0.5690 - val_loss: 1.1679 - val_precision: 0.7347 - val_recall: 0.5832
In [28]:
plt.plot(history_1.history['accuracy'])
plt.plot(history_1.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()

Evaluating the model on test data

In [29]:
loss,accuracy,recall,f1_score,precision = model.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 1s - 34ms/step - accuracy: 0.6547 - f1_score: 0.5761 - loss: 1.1621 - precision: 0.7242 - recall: 0.5916
In [30]:
print(f'Test Loss: {loss}')
print(f'Test Accuracy: {accuracy}')
print(f'Test Recall: {recall}')
print(f'Test f1_score: {f1_score}')
print(f'Test Precision: {precision}')
Test Loss: 1.1621086597442627
Test Accuracy: 0.6547368168830872
Test Recall: 0.5915789604187012
Test f1_score: [0.         0.7749999  0.57971007 0.87179476 0.23529407 0.5619834
 0.6823529  0.7906976  0.6804123  0.46153837 0.84313726 0.4313725 ]
Test Precision: 0.7242268323898315

Generating the predictions using test data

In [31]:
# Here we would get the output as probablities for each category
y_pred=model.predict(X_test_normalized)
15/15 ━━━━━━━━━━━━━━━━━━━━ 1s 22ms/step
In [32]:
y_pred
Out[32]:
array([[2.98052837e-12, 1.24485370e-11, 1.47089389e-16, ...,
        8.87103088e-04, 2.21332327e-12, 4.84843731e-05],
       [1.18972081e-23, 2.68033240e-03, 1.64843412e-04, ...,
        1.98282753e-04, 9.05292332e-01, 1.93197485e-02],
       [9.60499570e-22, 5.18237684e-05, 7.32219496e-06, ...,
        1.44425258e-05, 9.72269595e-01, 5.22493618e-03],
       ...,
       [2.93073177e-01, 8.40273518e-16, 4.69673078e-10, ...,
        9.05307226e-16, 1.00361212e-11, 2.20514848e-04],
       [4.92212382e-11, 6.25499297e-06, 4.34553300e-07, ...,
        5.27955703e-02, 1.12003997e-08, 1.11290012e-02],
       [5.72855818e-10, 1.18633181e-01, 1.86380371e-01, ...,
        1.21646270e-01, 1.73993319e-01, 1.06360994e-01]], dtype=float32)

Plotting the Confusion Matrix

In [33]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()
In [34]:
from sklearn.metrics import classification_report
In [35]:
# Generate classification report
report = classification_report(y_test_arg, y_pred_arg)
print(report)
              precision    recall  f1-score   support

           0       0.00      0.00      0.00        26
           1       0.76      0.79      0.78        39
           2       0.50      0.69      0.58        29
           3       0.91      0.84      0.87        61
           4       0.33      0.18      0.24        22
           5       0.47      0.71      0.56        48
           6       0.55      0.89      0.68        65
           7       0.81      0.77      0.79        22
           8       0.73      0.63      0.68        52
           9       0.56      0.39      0.46        23
          10       0.83      0.86      0.84        50
          11       0.85      0.29      0.43        38

    accuracy                           0.65       475
   macro avg       0.61      0.59      0.58       475
weighted avg       0.65      0.65      0.63       475

Observations

  • The Accuracy plot shows steady increase in training accuracy indicating model is learning well from training data

  • Validation accuracy improves but shows some fluctuations towards in end and t a slight decrease suggesting overfitting after 40th Epoch

  • Overall Accuracy is 0.65 which is moderate and can be improved .Recall (0.65), f1_score(0.63) and precision (0.65) can be improved too

CF Matrix/Classification Report

Class 0: Precision: 0.00, Recall: 0.00, F1-Score: 0.00, Support: 26 Indicates poor performance

Class 1: Precision: 0.76, Recall: 0.79, F1-Score: 0.78, Support: 39 Indicates good performance

Class 6: Precision: 0.55, Recall: 0.89, F1-Score: 0.68, Support: 65 High recall but moderate precision, indicating the model identifies most instances of Class 6 but includes some false positives

Model Performance Improvement

Reducing the Learning Rate:

Hint: Use ReduceLRonPlateau() function that will be used to decrease the learning rate by some factor, if the loss is not decreasing for some time. This may start decreasing the loss at a smaller learning rate. There is a possibility that the loss may still not decrease. This may lead to executing the learning rate reduction again in an attempt to achieve a lower loss.

Data Augmentation

Remember, data augmentation should not be used in the validation/test data set.

In [39]:
# Clearing backend
from tensorflow.keras import backend
backend.clear_session()

# Fixing the seed for random number generators
import random
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
In [40]:
# All images to be rescaled by 1/255.
train_datagen = ImageDataGenerator(
                              rotation_range=20,
                               width_shift_range=0.2,
                              height_shift_range=0.2,
                              shear_range=0.2,
                              zoom_range=0.2,
                              horizontal_flip=True,
                              fill_mode='nearest'
                              )
In [41]:
# Intializing a sequential model
model1 = Sequential()

# Adding first conv layer with 64 filters and kernel size 3x3 , padding 'same' provides the output size same as the input size
# Input_shape denotes input image dimension images
model1.add(Conv2D(64, (3, 3), activation='relu', padding="same", input_shape=(64, 64, 3)))

# Adding max pooling to reduce the size of output of first conv layer
model1.add(MaxPooling2D((2, 2), padding = 'same'))
# model.add(BatchNormalization())
model1.add(Conv2D(32, (3, 3), activation='relu', padding="same"))
model1.add(MaxPooling2D((2, 2), padding = 'same'))
model1.add(BatchNormalization())
# flattening the output of the conv layer after max pooling to make it ready for creating dense connections
model1.add(Flatten())

# Adding a fully connected dense layer with 100 neurons
model1.add(Dense(16, activation='relu'))
model1.add(Dropout(0.3))
# Adding the output layer with 12 neurons and activation functions as softmax since this is a multi-class classification problem
model1.add(Dense(12, activation='softmax'))

# Using SGD Optimizer
# opt = SGD(learning_rate=0.01, momentum=0.9)
opt=Adam()
# Compile model
model1.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy','recall','f1_score','precision'])

# Generating the summary of the model
model1.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ Layer (type)                          Output Shape                         Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ conv2d (Conv2D)                      │ (None, 64, 64, 64)          │           1,792 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d (MaxPooling2D)         │ (None, 32, 32, 64)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ conv2d_1 (Conv2D)                    │ (None, 32, 32, 32)          │          18,464 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ max_pooling2d_1 (MaxPooling2D)       │ (None, 16, 16, 32)          │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ batch_normalization                  │ (None, 16, 16, 32)          │             128 │
│ (BatchNormalization)                 │                             │                 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ flatten (Flatten)                    │ (None, 8192)                │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense (Dense)                        │ (None, 16)                  │         131,088 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dropout (Dropout)                    │ (None, 16)                  │               0 │
├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤
│ dense_1 (Dense)                      │ (None, 12)                  │             204 │
└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
 Total params: 151,676 (592.48 KB)
 Trainable params: 151,612 (592.23 KB)
 Non-trainable params: 64 (256.00 B)
In [42]:
from tensorflow.keras.callbacks import ReduceLROnPlateau
# Epochs
epochs = 50
# Batch size
batch_size = 32

reduce_lr = ReduceLROnPlateau(
    monitor='val_loss',    # Metric to monitor
    factor=0.1,            # Factor by which the learning rate will be reduced, new_lr = lr * factor
    patience=5,            # Number of epochs with no improvement after which learning rate will be reduced
    min_lr=0.00001,        # Lower bound on the learning rate
    verbose=1              # Verbosity mode
)
In [43]:
history_2 = model1.fit(train_datagen.flow(X_train_normalized,y_train_encoded,
                                       batch_size=batch_size,
                                       seed=42,
                                       shuffle=False),
                    epochs=epochs,
                    steps_per_epoch=X_train_normalized.shape[0] // batch_size,
                    validation_data=(X_val_normalized,y_val_encoded),callbacks=[reduce_lr],
                    verbose=1)
Epoch 1/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 13s 80ms/step - accuracy: 0.2010 - f1_score: 0.1144 - loss: 2.3993 - precision: 0.4717 - recall: 0.0102 - val_accuracy: 0.1432 - val_f1_score: 0.0296 - val_loss: 2.4455 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - learning_rate: 0.0010
Epoch 2/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 870us/step - accuracy: 0.1875 - f1_score: 0.1486 - loss: 2.4887 - precision: 1.0000 - recall: 0.0312 - val_accuracy: 0.1432 - val_f1_score: 0.0287 - val_loss: 2.4468 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - learning_rate: 0.0010
Epoch 3/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 5s 41ms/step - accuracy: 0.2727 - f1_score: 0.1432 - loss: 2.1659 - precision: 0.6168 - recall: 0.0525 - val_accuracy: 0.3811 - val_f1_score: 0.2165 - val_loss: 2.2739 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - learning_rate: 0.0010
Epoch 4/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.3125 - f1_score: 0.1588 - loss: 1.8869 - precision: 0.4000 - recall: 0.0625 - val_accuracy: 0.4084 - val_f1_score: 0.2270 - val_loss: 2.2574 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - learning_rate: 0.0010
Epoch 5/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 48ms/step - accuracy: 0.3350 - f1_score: 0.2141 - loss: 1.9718 - precision: 0.6627 - recall: 0.1337 - val_accuracy: 0.4147 - val_f1_score: 0.2633 - val_loss: 2.1213 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - learning_rate: 0.0010
Epoch 6/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.4062 - f1_score: 0.2246 - loss: 1.6590 - precision: 0.6667 - recall: 0.1875 - val_accuracy: 0.4189 - val_f1_score: 0.2627 - val_loss: 2.1230 - val_precision: 0.0000e+00 - val_recall: 0.0000e+00 - learning_rate: 0.0010
Epoch 7/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 9s 41ms/step - accuracy: 0.3477 - f1_score: 0.2300 - loss: 1.9314 - precision: 0.6213 - recall: 0.1358 - val_accuracy: 0.4274 - val_f1_score: 0.2636 - val_loss: 1.8184 - val_precision: 1.0000 - val_recall: 0.0547 - learning_rate: 0.0010
Epoch 8/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 894us/step - accuracy: 0.2812 - f1_score: 0.1761 - loss: 1.9692 - precision: 0.5556 - recall: 0.1562 - val_accuracy: 0.4189 - val_f1_score: 0.2619 - val_loss: 1.8363 - val_precision: 1.0000 - val_recall: 0.0526 - learning_rate: 0.0010
Epoch 9/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 54ms/step - accuracy: 0.3828 - f1_score: 0.2597 - loss: 1.8054 - precision: 0.6521 - recall: 0.1631 - val_accuracy: 0.4968 - val_f1_score: 0.3392 - val_loss: 1.6829 - val_precision: 0.8833 - val_recall: 0.1116 - learning_rate: 0.0010
Epoch 10/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.4375 - f1_score: 0.2601 - loss: 1.7668 - precision: 0.8000 - recall: 0.2500 - val_accuracy: 0.4526 - val_f1_score: 0.3074 - val_loss: 1.7436 - val_precision: 0.9600 - val_recall: 0.1011 - learning_rate: 0.0010
Epoch 11/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 9s 41ms/step - accuracy: 0.4141 - f1_score: 0.2884 - loss: 1.6892 - precision: 0.6657 - recall: 0.1960 - val_accuracy: 0.4126 - val_f1_score: 0.2767 - val_loss: 1.7429 - val_precision: 0.5556 - val_recall: 0.2632 - learning_rate: 0.0010
Epoch 12/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5312 - f1_score: 0.3755 - loss: 1.4026 - precision: 0.8889 - recall: 0.2500 - val_accuracy: 0.4695 - val_f1_score: 0.3244 - val_loss: 1.5356 - val_precision: 0.6505 - val_recall: 0.2547 - learning_rate: 0.0010
Epoch 13/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 48ms/step - accuracy: 0.4365 - f1_score: 0.3137 - loss: 1.6487 - precision: 0.6616 - recall: 0.2160 - val_accuracy: 0.3895 - val_f1_score: 0.2492 - val_loss: 1.9649 - val_precision: 0.5000 - val_recall: 0.2905 - learning_rate: 0.0010
Epoch 14/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 841us/step - accuracy: 0.4062 - f1_score: 0.2095 - loss: 1.7511 - precision: 0.8182 - recall: 0.2812 - val_accuracy: 0.3453 - val_f1_score: 0.1952 - val_loss: 2.3383 - val_precision: 0.4313 - val_recall: 0.2905 - learning_rate: 0.0010
Epoch 15/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 5s 40ms/step - accuracy: 0.4579 - f1_score: 0.3290 - loss: 1.5476 - precision: 0.6858 - recall: 0.2362 - val_accuracy: 0.4863 - val_f1_score: 0.3830 - val_loss: 1.4101 - val_precision: 0.7891 - val_recall: 0.2442 - learning_rate: 0.0010
Epoch 16/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 840us/step - accuracy: 0.6875 - f1_score: 0.4520 - loss: 1.4846 - precision: 0.7500 - recall: 0.2812 - val_accuracy: 0.4758 - val_f1_score: 0.3658 - val_loss: 1.4386 - val_precision: 0.7961 - val_recall: 0.2547 - learning_rate: 0.0010
Epoch 17/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 53ms/step - accuracy: 0.4726 - f1_score: 0.3733 - loss: 1.5235 - precision: 0.7048 - recall: 0.2490 - val_accuracy: 0.4905 - val_f1_score: 0.3644 - val_loss: 1.4951 - val_precision: 0.7013 - val_recall: 0.3411 - learning_rate: 0.0010
Epoch 18/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 876us/step - accuracy: 0.4688 - f1_score: 0.2844 - loss: 1.4914 - precision: 0.8750 - recall: 0.2188 - val_accuracy: 0.4484 - val_f1_score: 0.3402 - val_loss: 1.6179 - val_precision: 0.6293 - val_recall: 0.3074 - learning_rate: 0.0010
Epoch 19/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 5s 41ms/step - accuracy: 0.4819 - f1_score: 0.3803 - loss: 1.4877 - precision: 0.6822 - recall: 0.2667 - val_accuracy: 0.5874 - val_f1_score: 0.4658 - val_loss: 1.1879 - val_precision: 0.7566 - val_recall: 0.4253 - learning_rate: 0.0010
Epoch 20/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 965us/step - accuracy: 0.5625 - f1_score: 0.3558 - loss: 1.7148 - precision: 0.7333 - recall: 0.3438 - val_accuracy: 0.6063 - val_f1_score: 0.4825 - val_loss: 1.0999 - val_precision: 0.8134 - val_recall: 0.4589 - learning_rate: 0.0010
Epoch 21/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 51ms/step - accuracy: 0.4983 - f1_score: 0.4010 - loss: 1.4541 - precision: 0.7010 - recall: 0.2954 - val_accuracy: 0.5305 - val_f1_score: 0.4103 - val_loss: 1.2906 - val_precision: 0.7644 - val_recall: 0.3621 - learning_rate: 0.0010
Epoch 22/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.4688 - f1_score: 0.3264 - loss: 1.4739 - precision: 0.6000 - recall: 0.2812 - val_accuracy: 0.5558 - val_f1_score: 0.4276 - val_loss: 1.2465 - val_precision: 0.7661 - val_recall: 0.4000 - learning_rate: 0.0010
Epoch 23/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 9s 41ms/step - accuracy: 0.4920 - f1_score: 0.3943 - loss: 1.4495 - precision: 0.7184 - recall: 0.3017 - val_accuracy: 0.5937 - val_f1_score: 0.4923 - val_loss: 1.1230 - val_precision: 0.7179 - val_recall: 0.4821 - learning_rate: 0.0010
Epoch 24/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 965us/step - accuracy: 0.5312 - f1_score: 0.3720 - loss: 1.4317 - precision: 0.6842 - recall: 0.4062 - val_accuracy: 0.5958 - val_f1_score: 0.4851 - val_loss: 1.1298 - val_precision: 0.7255 - val_recall: 0.4674 - learning_rate: 0.0010
Epoch 25/50
117/118 ━━━━━━━━━━━━━━━━━━━ 0s 53ms/step - accuracy: 0.5104 - f1_score: 0.4285 - loss: 1.4150 - precision: 0.6841 - recall: 0.3247
Epoch 25: ReduceLROnPlateau reducing learning rate to 0.00010000000474974513.
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 54ms/step - accuracy: 0.5104 - f1_score: 0.4286 - loss: 1.4146 - precision: 0.6843 - recall: 0.3246 - val_accuracy: 0.5389 - val_f1_score: 0.4417 - val_loss: 1.3466 - val_precision: 0.6395 - val_recall: 0.4295 - learning_rate: 0.0010
Epoch 26/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 907us/step - accuracy: 0.4062 - f1_score: 0.3174 - loss: 1.3448 - precision: 0.5714 - recall: 0.2500 - val_accuracy: 0.5389 - val_f1_score: 0.4426 - val_loss: 1.3460 - val_precision: 0.6364 - val_recall: 0.4274 - learning_rate: 1.0000e-04
Epoch 27/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 8s 41ms/step - accuracy: 0.5252 - f1_score: 0.4416 - loss: 1.3687 - precision: 0.6917 - recall: 0.3418 - val_accuracy: 0.6400 - val_f1_score: 0.5574 - val_loss: 1.0579 - val_precision: 0.8297 - val_recall: 0.4821 - learning_rate: 1.0000e-04
Epoch 28/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 862us/step - accuracy: 0.3750 - f1_score: 0.2584 - loss: 1.6384 - precision: 0.6429 - recall: 0.2812 - val_accuracy: 0.6400 - val_f1_score: 0.5572 - val_loss: 1.0498 - val_precision: 0.8297 - val_recall: 0.4821 - learning_rate: 1.0000e-04
Epoch 29/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 7s 55ms/step - accuracy: 0.5454 - f1_score: 0.4697 - loss: 1.2912 - precision: 0.7476 - recall: 0.3620 - val_accuracy: 0.7032 - val_f1_score: 0.6417 - val_loss: 0.9618 - val_precision: 0.8153 - val_recall: 0.5389 - learning_rate: 1.0000e-04
Epoch 30/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 942us/step - accuracy: 0.6250 - f1_score: 0.4803 - loss: 1.0564 - precision: 0.9333 - recall: 0.4375 - val_accuracy: 0.7032 - val_f1_score: 0.6446 - val_loss: 0.9616 - val_precision: 0.8179 - val_recall: 0.5389 - learning_rate: 1.0000e-04
Epoch 31/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 9s 47ms/step - accuracy: 0.5520 - f1_score: 0.4656 - loss: 1.2638 - precision: 0.7218 - recall: 0.3718 - val_accuracy: 0.6484 - val_f1_score: 0.5782 - val_loss: 1.1147 - val_precision: 0.7452 - val_recall: 0.4863 - learning_rate: 1.0000e-04
Epoch 32/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5938 - f1_score: 0.4329 - loss: 1.0797 - precision: 0.8824 - recall: 0.4688 - val_accuracy: 0.6463 - val_f1_score: 0.5726 - val_loss: 1.1303 - val_precision: 0.7395 - val_recall: 0.4842 - learning_rate: 1.0000e-04
Epoch 33/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 9s 41ms/step - accuracy: 0.5597 - f1_score: 0.4782 - loss: 1.2157 - precision: 0.7419 - recall: 0.3879 - val_accuracy: 0.6232 - val_f1_score: 0.5419 - val_loss: 1.1325 - val_precision: 0.7220 - val_recall: 0.4758 - learning_rate: 1.0000e-04
Epoch 34/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.4375 - f1_score: 0.2757 - loss: 1.2407 - precision: 0.6111 - recall: 0.3438 - val_accuracy: 0.6253 - val_f1_score: 0.5484 - val_loss: 1.1275 - val_precision: 0.7244 - val_recall: 0.4758 - learning_rate: 1.0000e-04
Epoch 35/50
117/118 ━━━━━━━━━━━━━━━━━━━ 0s 52ms/step - accuracy: 0.5799 - f1_score: 0.5134 - loss: 1.2489 - precision: 0.7467 - recall: 0.3881
Epoch 35: ReduceLROnPlateau reducing learning rate to 1.0000000474974514e-05.
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 53ms/step - accuracy: 0.5797 - f1_score: 0.5130 - loss: 1.2492 - precision: 0.7466 - recall: 0.3879 - val_accuracy: 0.6211 - val_f1_score: 0.5471 - val_loss: 1.2339 - val_precision: 0.7092 - val_recall: 0.4568 - learning_rate: 1.0000e-04
Epoch 36/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5625 - f1_score: 0.3607 - loss: 1.0300 - precision: 0.7368 - recall: 0.4375 - val_accuracy: 0.6211 - val_f1_score: 0.5471 - val_loss: 1.2338 - val_precision: 0.7092 - val_recall: 0.4568 - learning_rate: 1.0000e-05
Epoch 37/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 5s 41ms/step - accuracy: 0.5705 - f1_score: 0.4858 - loss: 1.2561 - precision: 0.7534 - recall: 0.3750 - val_accuracy: 0.6737 - val_f1_score: 0.6108 - val_loss: 1.0172 - val_precision: 0.7890 - val_recall: 0.5116 - learning_rate: 1.0000e-05
Epoch 38/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 926us/step - accuracy: 0.7188 - f1_score: 0.5998 - loss: 1.0652 - precision: 0.8750 - recall: 0.4375 - val_accuracy: 0.6695 - val_f1_score: 0.6069 - val_loss: 1.0154 - val_precision: 0.7922 - val_recall: 0.5137 - learning_rate: 1.0000e-05
Epoch 39/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 6s 49ms/step - accuracy: 0.5648 - f1_score: 0.4970 - loss: 1.2535 - precision: 0.7544 - recall: 0.3702 - val_accuracy: 0.6779 - val_f1_score: 0.6140 - val_loss: 0.9973 - val_precision: 0.8046 - val_recall: 0.5200 - learning_rate: 1.0000e-05
Epoch 40/50
  1/118 ━━━━━━━━━━━━━━━━━━━━ 1s 9ms/step - accuracy: 0.5625 - f1_score: 0.3282 - loss: 1.2369 - precision: 0.6875 - recall: 0.3438
Epoch 40: ReduceLROnPlateau reducing learning rate to 1e-05.
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.5625 - f1_score: 0.3282 - loss: 1.2369 - precision: 0.6875 - recall: 0.3438 - val_accuracy: 0.6779 - val_f1_score: 0.6140 - val_loss: 0.9977 - val_precision: 0.8046 - val_recall: 0.5200 - learning_rate: 1.0000e-05
Epoch 41/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 9s 40ms/step - accuracy: 0.5727 - f1_score: 0.4998 - loss: 1.2405 - precision: 0.7428 - recall: 0.3904 - val_accuracy: 0.6611 - val_f1_score: 0.5981 - val_loss: 1.0266 - val_precision: 0.7853 - val_recall: 0.5158 - learning_rate: 1.0000e-05
Epoch 42/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 878us/step - accuracy: 0.6875 - f1_score: 0.4903 - loss: 0.9163 - precision: 0.8095 - recall: 0.5312 - val_accuracy: 0.6611 - val_f1_score: 0.5984 - val_loss: 1.0273 - val_precision: 0.7846 - val_recall: 0.5137 - learning_rate: 1.0000e-05
Epoch 43/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 7s 54ms/step - accuracy: 0.5647 - f1_score: 0.4814 - loss: 1.2532 - precision: 0.7392 - recall: 0.3870 - val_accuracy: 0.6674 - val_f1_score: 0.6055 - val_loss: 1.0273 - val_precision: 0.7827 - val_recall: 0.5158 - learning_rate: 1.0000e-05
Epoch 44/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 873us/step - accuracy: 0.5938 - f1_score: 0.4841 - loss: 1.1697 - precision: 0.7619 - recall: 0.5000 - val_accuracy: 0.6674 - val_f1_score: 0.6055 - val_loss: 1.0270 - val_precision: 0.7853 - val_recall: 0.5158 - learning_rate: 1.0000e-05
Epoch 45/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 5s 41ms/step - accuracy: 0.5583 - f1_score: 0.4793 - loss: 1.2378 - precision: 0.7402 - recall: 0.3783 - val_accuracy: 0.6674 - val_f1_score: 0.6011 - val_loss: 1.0022 - val_precision: 0.7994 - val_recall: 0.5200 - learning_rate: 1.0000e-05
Epoch 46/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.5312 - f1_score: 0.3647 - loss: 1.1870 - precision: 0.6842 - recall: 0.4062 - val_accuracy: 0.6653 - val_f1_score: 0.5987 - val_loss: 1.0019 - val_precision: 0.8000 - val_recall: 0.5221 - learning_rate: 1.0000e-05
Epoch 47/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 10s 41ms/step - accuracy: 0.5593 - f1_score: 0.4924 - loss: 1.2734 - precision: 0.7227 - recall: 0.3655 - val_accuracy: 0.6737 - val_f1_score: 0.6070 - val_loss: 0.9769 - val_precision: 0.8176 - val_recall: 0.5284 - learning_rate: 1.0000e-05
Epoch 48/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 873us/step - accuracy: 0.6875 - f1_score: 0.5872 - loss: 1.0838 - precision: 0.8500 - recall: 0.5312 - val_accuracy: 0.6737 - val_f1_score: 0.6070 - val_loss: 0.9777 - val_precision: 0.8182 - val_recall: 0.5305 - learning_rate: 1.0000e-05
Epoch 49/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 5s 41ms/step - accuracy: 0.5887 - f1_score: 0.5127 - loss: 1.1793 - precision: 0.7588 - recall: 0.4099 - val_accuracy: 0.6758 - val_f1_score: 0.6110 - val_loss: 0.9861 - val_precision: 0.8117 - val_recall: 0.5263 - learning_rate: 1.0000e-05
Epoch 50/50
118/118 ━━━━━━━━━━━━━━━━━━━━ 0s 1ms/step - accuracy: 0.6250 - f1_score: 0.4485 - loss: 1.3209 - precision: 0.7368 - recall: 0.4375 - val_accuracy: 0.6758 - val_f1_score: 0.6112 - val_loss: 0.9868 - val_precision: 0.8117 - val_recall: 0.5263 - learning_rate: 1.0000e-05
In [44]:
plt.plot(history_2.history['accuracy'])
plt.plot(history_2.history['val_accuracy'])
plt.title('Model Accuracy')
plt.ylabel('Accuracy')
plt.xlabel('Epoch')
plt.legend(['Train', 'Validation'], loc='upper left')
plt.show()
In [48]:
accuracy = model1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - 6ms/step - accuracy: 0.6611 - f1_score: 0.6039 - loss: 1.0477 - precision: 0.7789 - recall: 0.4968
In [47]:
loss,accuracy,recall,f1_score,precision = model1.evaluate(X_test_normalized, y_test_encoded, verbose=2)
15/15 - 0s - 6ms/step - accuracy: 0.6611 - f1_score: 0.6039 - loss: 1.0477 - precision: 0.7789 - recall: 0.4968
In [49]:
# Here we would get the output as probablities for each category
y_pred=model1.predict(X_test_normalized)
15/15 ━━━━━━━━━━━━━━━━━━━━ 1s 23ms/step
In [50]:
# Obtaining the categorical values from y_test_encoded and y_pred
y_pred_arg=np.argmax(y_pred,axis=1)
y_test_arg=np.argmax(y_test_encoded,axis=1)

# Plotting the Confusion Matrix using confusion matrix() function which is also predefined tensorflow module
confusion_matrix = tf.math.confusion_matrix(y_test_arg,y_pred_arg)
f, ax = plt.subplots(figsize=(10, 8))
sns.heatmap(
    confusion_matrix,
    annot=True,
    linewidths=.4,
    fmt="d",
    square=True,
    ax=ax
)
plt.show()
In [51]:
# Generate classification report
report2 = classification_report(y_test_arg, y_pred_arg)
print(report2)
              precision    recall  f1-score   support

           0       0.12      0.04      0.06        26
           1       0.51      0.92      0.65        39
           2       0.59      0.55      0.57        29
           3       0.89      0.90      0.89        61
           4       0.70      0.32      0.44        22
           5       0.81      0.52      0.63        48
           6       0.67      0.71      0.69        65
           7       0.74      0.91      0.82        22
           8       0.67      0.60      0.63        52
           9       0.80      0.35      0.48        23
          10       0.85      0.80      0.82        50
          11       0.43      0.76      0.55        38

    accuracy                           0.66       475
   macro avg       0.65      0.61      0.60       475
weighted avg       0.67      0.66      0.65       475

Observations

  • The Training accuracy shows steady increase reaching upto 66%.Validation accuracy increases more rapidly and stabilises indicating model is generalising well

  • CF : Most of the values are concentrated along diagonals indicating good performance with many correct predictions,however model tends to confuse certain classes with others

  • Classes 1,3,6,7 show strong performance with high precision and recall.Class 0 shows the poorest perfoamnce

Final Model

Comment on the final model you have selected and use the same in the below code to visualize the image.

In [63]:
pd.DataFrame({'Models':['Base CNN Model','CNN Model with Data Augmentation'], 'Accuracy':['65%','66%'],'Recall':['65%','66%',],'Precision':['65%','67%'],'F1_Score':['63%','65%']})
Out[63]:
Models Accuracy Recall Precision F1_Score
0 Base CNN Model 65% 65% 65% 63%
1 CNN Model with Data Augmentation 66% 66% 67% 65%

Final Model Selection

Comparison and Recommendation

  • Accuracy: The CNN Model with Data Augmentation has a slight edge with 66% compared to 65% for the Base CNN Model.

  • Recall: Similar improvement with the Data Augmentation model showing 66% recall.

  • Precision: Higher precision of 67% for the Data Augmentation model versus 65% for the Base model.

  • F1 Score: An improvement to 65% from 63%.

Conclusion

Based on these performance metrics, the CNN Model with Data Augmentation demonstrates overall better performance across all evaluated metrics (accuracy, recall, precision, and F1 score). Therefore, the CNN Model with Data Augmentation is the better model and is recommended for further use.

Visualizing the prediction

In [52]:
# Visualizing the predicted and correct label of images from test data
plt.figure(figsize=(2,2))
plt.imshow(X_test[2])
plt.show()
print('Predicted Label', enc.inverse_transform(model1.predict((X_test_normalized[2].reshape(1,64,64,3)))))   # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[2])                                               # using inverse_transform() to get the output label from the output vector
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 463ms/step
Predicted Label ['Small-flowered Cranesbill']
True Label Small-flowered Cranesbill
In [53]:
plt.figure(figsize=(2,2))
plt.imshow(X_test[33])
plt.show()
print('Predicted Label', enc.inverse_transform(model1.predict((X_test_normalized[33].reshape(1,64,64,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[33])                                              # using inverse_transform() to get the output label from the output vector
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 28ms/step
Predicted Label ['Cleavers']
True Label Cleavers
In [54]:
plt.figure(figsize=(2,2))
plt.imshow(X_test[36])
plt.show()
print('Predicted Label', enc.inverse_transform(model1.predict((X_test_normalized[36].reshape(1,64,64,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[36])
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 29ms/step
Predicted Label ['Shepherds Purse']
True Label Shepherds Purse
In [55]:
plt.figure(figsize=(2,2))
plt.imshow(X_test[5])
plt.show()
print('Predicted Label', enc.inverse_transform(model1.predict((X_test_normalized[36].reshape(1,64,64,3)))))  # reshaping the input image as we are only trying to predict using a single image
print('True Label', enc.inverse_transform(y_test_encoded)[5])
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 46ms/step
Predicted Label ['Shepherds Purse']
True Label Small-flowered Cranesbill

Observation

  • We can see from above that our chosen model has correct;y predicted labels in 3 out of 4 instances, which is a decent prediction level

Actionable Insights and Business Recommendations

Actionable Insights

  1. Overfitting Reduction : By using techniques like regularisation, dropout,Early stopping to improve model generalisation.

  2. Data Augmenttaion and Hyperparamemeter Tuning : Fine Tune the model by using enhanced Data Augmentation and hyper paramater techniques.

  3. Model Performance : Should be further enhanced to increase predictive performance. Also class imbalance should be addressed in minority classes by applying over/undersmpling and weighted class techniques.

BusinessRecommendations

  1. Build and deploy an AI powered seedling identification system for farmers & agricultarists which can enable real-time identification saving a great amount of manual labor and cost. Integrating this with mobile/drone devices can enable remote monitoring and seedling identification in large fields.

  2. Use the insights from the classification system for optimum resource allocations like water,fertilisers,pestisides leading to savaning and sustainable farming practices.

  3. Use the classification system for better plan identification system enabling better crop management and targeted agricultural practices.